Learning to Segment From Scribbles Using Multi-Scale Adversarial Attention Gates

Large, fine-grained image segmentation datasets, annotated at pixel-level, are difficult to obtain, particularly in medical imaging, where annotations also require expert knowledge. Weakly-supervised learning can train models by relying on weaker forms of annotation, such as scribbles. Here, we learn segment using scribble an adversarial game. With unpaired masks, a multi-scale GAN generate realistic masks multiple resolutions, while use scribbles their correct position the image. Central model's success is novel attention gating mechanism, which condition with signals act shape prior, resulting better object localization scales. Subject conditioning, segmentor learns maps that semantic, suppress noisy activations outside objects, and reduce vanishing gradient problem deeper layers segmentor. We evaluated our model several (ACDC, LVSC, CHAOS) non-medical (PPSS) report performance levels matching those achieved trained fully masks. demonstrate extensions variety settings: semi-supervised learning; combining sources (a crowdsourcing scenario) multi-task (combining mask supervision). release expert-made for ACDC dataset, code used experiments, https://vios-s.github.io/multiscale-adversarial-attention-gates.